Learning Rewrite Rules for Search Database Systems Using Query Logs

Recent literature on “search database systems” has introduced the notion of using query rewrite rules to influence the behavior of a search engine. Rewrite rules enable domain experts and search administrators to customize the search engine by providing a powerful rule-driven framework to transform user search queries. In this paper, we address the important problem of automatically learning such query rewrite rules from query logs using a novel Hidden Markov Model (HMM) formulation. Our formulation captures both the latent information present in the sequence of queries issued within a session as well as the explicit information in the form of user clicks on individual results. We have developed the notion of support and confidence for query rewrite rules and leveraged these concepts to harvest high quality rewrite rules from the trained HMM. We propose rigorous evaluation schemes to quantify the efficacy of a set of rewrite rules and demonstrate the effectiveness of our approach through extensive experiments using the publicly available AOL query log data set and the YAGO concept ontology.